Searching with Tags: Do Tags Help Users Find Things?

This study examines the question of whether tags can be useful in the process of information retrieval. Participants searched a social bookmarking tool specialising in academic articles (CiteULike) and an online journal database (Pubmed). Participant actions were captured using screen capture software and they were asked to describe their search process. Users did make use of tags in their search process, as a guide to searching and as hyperlinks to potentially useful articles. However, users also made use of controlled vocabularies in the journal database to locate useful search terms and of links to related articles supplied by the database

Evaluation of Mappings from MARC to Linked Data

The purpose of this study is to assess the quality and compatibility of library linked data (LLD) schemas in use or proposed for library resources. Linked Data (LD) has the potential to provide high quality metadata on the web with the ability to incorporate existing structured data from MARC via a mapping.  Researchers selected representative libraries such as Harvard University Library, LC BIBFRAME (Library of Congress Bibliographic Framework), OCLC (Online Computer Library Canter) WorldCat, and National Library of Spain. For LD frameworks, four resources are matched into specific categories with MARC (MAchine-Readable Cataloging) tags so that it could be retrieved in both OCLC LD and BIBFRAME with the conversion tool at bibframe.org: (1) Classic, ebook,and fiction, (2) multiple authors and part of a series, and non-fiction, (3) varying title, translation, and fiction, and (4) sub title, non-fiction. This study revealed that the choices and elements of each library made in local decisions might bring interoperability issues for LD services due to the quality metadata creation issues

Tagging for Health Information Organization and Retrieval

This paper examines the tagging practices evident on CiteULike, a research oriented social bookmarking site for journal articles. Articles selected for this study were health information and medicine related. Tagging practices were examined using standard informetric measures for analysis of bibliographic information and analysis of term use. Additionally, tags were compared to descriptors assigned to the same article

User, Author and Professional Indexing in Context: An Exploration of Tagging Practices on CiteULike

This paper examines the context of online indexing from the viewpoint of three different groups: users, authors, and intermediaries. User, author and intermediary keywords were collected from journal articles tagged on CiteULike and analysed. Descriptive statistics, informetric measures, and thesaural term comparison shows that there are important differences in the context of keywords from the three groups

Convergence and divergence in tagging systems: An examination of tagging practices over a four year period

This paper analyses the tagging patterns on delicious.com for a set of documents (URLs) over a 4 year period using informetrics methods to assess how collaborative tagging supports and enhances traditional document indexing. Results of the study show that there is still a mix of consensus and divergence in tagging term use and tagging patterns. While some of the chosen URLs maintained or even increased their popularity, others experienced a severe drop in popularity. Tag usage showed some measures of influence between taggers, but users continued to use their own tags and the proportions of unique and untagged items has increased. Proportions of tags used showed fluctuations with islands of stability over a 4 year period. Patterns in the usage of tags assigned to these documents also highlighted practices related to personal and collective information organisation which conventional systems are unable to facilitate

Tagging and Findability: Do Tags Help Users Find Things?

Background The question of findability is an old one and led directly to the creation of cataloguing and classification systems for the organisation of knowledge. However, such systems have not proven to be truly scalable when dealing with digital information and especially information on the web. Can the user created categories and classification schemes of tagging be used to enhance findability in these new environments? Much speculation has been advanced on the subject but so far no empirical studies have been done. This study looks at the use of tags as an aid to findability. In essence, this is an examination of some of the questions of classic information retrieval research in a new context. Proponents of tagging often suggest that tags could provide at worst an entry vocabulary for traditional classification systems and at best a complete replacement for such systems. One method for judging the usefulness of a classification system for enabling retrieval is to perform an information retrieval study on the system with volunteer searchers. Many such experiments have been conducted in the library science literature and increasingly these studies are performed on web search engines as more and more users search the web first. Studies by Leighton and Srivastava (1999), Clarke and Willet (1997), Su (2003b) all evaluated a variety of search engines using standard evaluation methods from earlier non web information retrieval systems such as recall and precision, but many also suggested alternative methods for evaluating the effectiveness of a search engine given that recall and precision cannot truly be calculated. (Tang and Sun 2003; Vaughan 2004) A key component in all modern information retrieval studies is the relevance judgements of users of the retrieval system. (Cosijn and Ingwersen 2000; Tang and Sun 2003; Oppenheim et al. 2000) Therefore, this study would place a great emphasis on the relevance judgements of the participants. As noted above, relevance judgements are often used to enable the calculation of metrics for evaluating the effectiveness of information retrieval systems. In all studies examining information retrieval systems using keyword or index term searches, there is an implicit evaluation of the effectiveness of classification terms. In evaluating classification terms, it is important to evaluate not only the retrieval effectiveness of the search term, in terms of how many relevant documents were retrieved, but also in terms of how long it took the user to think of using this term in this context and whether or not the user thought the term was appropriate or useful for the document. One way to examine the potential uses of tags in the search process would be to compare the search experience between social bookmarking tools and other methods of information retrieval such as retrieval via controlled vocabulary or retrieval via free text search. Research Questions Do tags appear to enhance findability? Do users feel that they have found what they are looking for? How do users find searching social bookmarking sites compared to searching more classically organised sites? Do users think that tags assigned by other users are more intuitive? Do tagging structures facilitate information retrieval? How does this compare to traditional structures of supporting information retrieval? Methodology and Analysis In order to compare the usability of tags versus traditional classification schemes, a study will be conducted using volunteer searchers. These searchers would be asked to search an electronic journal database and a social bookmarking site for information on a specific topic. Screen capture software, a think aloud protocol and an exit interview will be used to capture the impressions of the users when faced with traditional classification or user tags and their usefulness in the search process. While information concerning the usability of the systems themselves for searching may be of interest, data collection will be focused on a comparison of the terms entered by the participants. A sample size of 20 participants for the study should allow for the collection of data until saturation is reached, in other words until no further new information is being collected. Since this is an exploratory study, 20 participants should provide a good selection for the collection of data. Each participant will search for information using both the traditional on-line database with assigned descriptors and a social bookmarking site. This study will ask users to use pubmed (an on-line database) in comparison to citeulike (a social bookmarking tool) to find academic articles on given topics. Participants will be asked to perform the searches in the order specified so that their use of a social bookmarking site first versus an on-line database can be randomised. Participants will be divided into two groups. One group will search the social bookmarking tool first while the other group will use the on-line database first. Participants will be assigned a topic for which they will search for articles. These topics will be articulated as short paragraphs describing an information need. Participants will select their own keywords for both the on-line database and the social bookmarking tools and then provide relevance judgements of the results. Initially, participants will be asked to provide a list of words they consider relevant as an initial search set. Each search will produce a result set of bibliographic information. The participant will be asked to select the top 10 articles from the result set and assign a relevance score to the article based on an examination of the available information. The available information may include the title, tags and abstract as well as the full text of the article via links to an on-line database. At the end of each search, participants will be asked to make a list of what terms they would now use if asked to search for this information again. Participants will not have access to their initial set of search terms at this time to eliminate the learning effect. Three sets of data will be available for analysis: sets of initial and final keywords selected by the user, the recording of the search session and think aloud, and recorded exit interviews after the search session. Each set of data can be analysed to examine user impressions of the search process from the perspective of the keywords (tags or index terms respectively). Both traditional and newly suggested measures for evaluating information retrieval systems will be used to compare the two systems on standard efficiency measures. Additionally, keywords and tags chosen by users will be compared and examined to see how they are related (or not). Keywords and tags assigned to articles judged relevant by the users will be compared to those used for articles judged to be less relevant or not relevant. It is expected that this preliminary study will provide some insights into user's choices of preliminary keywords for searching. One of the questions that users will be asked is to describe how they selected their initial search terms. They will also be asked to compare their initial search terms to their final search terms

Tagging of Biomedical Articles on CiteULike: A Comparison of User, Author and Professional Indexing

This paper examines the context of online indexing from the viewpoint of three different groups: users, authors, and professional indexers. User tags, author keywords and descriptors were collected from academic journal articles, which were both indexed in Pubmed and tagged on CiteULike, and analysed. Descriptive statistics, informetric measures, and thesaural term comparison shows that there are important differences in the use of keywords between the three groups in addition to similarities which can be used to enhance support for search and browse. While tags and author keywords were found that matched descriptors exactly, other terms which did not match but provided important expansion to the indexing lexicon were found. These additional terms could be used to enhance support for searching and browsing in article databases as well as to provide invaluable data for entry vocabulary and emergent terminology for regular updates to indexing systems. Additionally, the study suggests that tags support organisation by association to task, projects and subject while making important connections to traditional systems which classify into subject categories

Searching with Tags: Do Tags Help Users Find Things?

This study examines the question of whether tags can be useful in the process of information retrieval. Participants were asked to search a social bookmarking tool specialising in academic articles (CiteULike) and an online journal database (Pubmed) in order to determine if users found tags were useful in their search process. The actions of each participants were captured using screen capture software and they were asked to describe their search process. The preliminary study showed that users did indeed make use of tags in their search process, as a guide to searching and as hyperlinks to potentially useful articles. However, users also made use of controlled vocabularies in the journal database

Information Organisation Practices on the Web: Tagging and the Social Organisation of Information

This talk (the public talk for my thesis) examines the phenomenon of social tagging from its early beginnings to its current level of prominence on a wide variety of websites in a series of linked studies examining the structures and patterns of tag term use to determine whether regular patterns appear that would support information organisation and retrieval